Accelerating Training of Deep Neural Networks via Sparse Edge Processing
نویسندگان
چکیده
We propose a reconfigurable hardware architecture for deep neural networks (DNNs) capable of online training and inference, which uses algorithmically pre-determined, structured sparsity to significantly lower memory and computational requirements. This novel architecture introduces the notion of edge-processing to provide flexibility and combines junction pipelining and operational parallelization to speed up training. The overall effect is to reduce network complexity by factors up to 30x and training time by up to 35x relative to GPUs, while maintaining high fidelity of inference results. This has the potential to enable extensive parameter searches and development of the largely unexplored theoretical foundation of DNNs. The architecture automatically adapts itself to different network sizes given available hardware resources. As proof of concept, we show results obtained for different bit widths.
منابع مشابه
Quad-pixel edge detection using neural network
One of the most fundamental features of digital image and the basic steps in image processing, analysis, pattern recognition and computer vision is the edge of an image where the preciseness and reliability of its results will affect directly on the comprehension machine system made objective world. Several edge detectors have been developed in the past decades, although no single edge detector...
متن کاملQuad-pixel edge detection using neural network
One of the most fundamental features of digital image and the basic steps in image processing, analysis, pattern recognition and computer vision is the edge of an image where the preciseness and reliability of its results will affect directly on the comprehension machine system made objective world. Several edge detectors have been developed in the past decades, although no single edge detector...
متن کاملCystoscopy Image Classication Using Deep Convolutional Neural Networks
In the past three decades, the use of smart methods in medical diagnostic systems has attractedthe attention of many researchers. However, no smart activity has been provided in the eld ofmedical image processing for diagnosis of bladder cancer through cystoscopy images despite the highprevalence in the world. In this paper, two well-known convolutional neural networks (CNNs) ...
متن کاملDeep Rewiring: Training very sparse deep networks
Neuromorphic hardware tends to pose limits on the connectivity of deep networks that one can run on them. But also generic hardware and software implementations of deep learning run more efficiently on sparse networks. Several methods exist for pruning connections of a neural network after it was trained without connectivity constraints. We present an algorithm, DEEP R, that enables us to train...
متن کاملA Hybrid Optimization Algorithm for Learning Deep Models
Deep learning is one of the subsets of machine learning that is widely used in Artificial Intelligence (AI) field such as natural language processing and machine vision. The learning algorithms require optimization in multiple aspects. Generally, model-based inferences need to solve an optimized problem. In deep learning, the most important problem that can be solved by optimization is neural n...
متن کامل